Rendezvousing resource requests with corresponding resources

ABSTRACT

The present invention extends to methods, systems, and computer program products for rendezvousing resource requests with corresponding resources. Doubly linked sorted lists are traversed using modulo arithmetic in both directions. Sorted lists can be partitioned based on a multiple proximity metrics. Node routing tables provide a logarithmic index to nodes within the ID space of the federation infrastructure to facilitate more efficient routing. Messages can be routed to nodes within a ring and proximally routed to nodes in other partitioned rings.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.10/971,451, filed Oct. 22, 2004, and entitled “Rendezvousing ResourceRequests with Corresponding Resources,” which is herein incorporated byreference in its entirety.

BACKGROUND OF THE INVENTION

1. The Field of the Invention

The present invention relates to accessing resources and, moreparticularly, to rendezvousing resource requests with correspondingresources.

2. Background and Relevant Art

Computer systems and related technology affect many aspects of society.Indeed, the computer system's ability to process information hastransformed the way we live and work. Computer systems now commonlyperform a host of tasks (e.g., word processing, scheduling, and databasemanagement) that prior to the advent of the computer system wereperformed manually. More recently, computer systems have been coupled toone another and to other electronic devices to form both wired andwireless computer networks over which the computer systems and otherelectronic devices can transfer electronic data. As a result, many tasksperformed at a computer system (e.g., voice communication, accessingelectronic mail, controlling home electronics, Web browsing, andprinting documents) include electronic communication between a number ofcomputer systems and/or other electronic devices via wired and/orwireless computer networks.

However, to utilize a network resource to perform a computerized task, acomputer system must have some way to identify and access the networkresource. Accordingly, resources are typically assigned uniqueidentifiers, for example, network addresses, that uniquely identifyresources and can be used to distinguish one resource from otherresources. Thus, a computer system that desires to utilize a resourcecan connect to the resource using the network address that correspondsto the resource. However, accessing a network resource can be difficultif a computer system has no prior knowledge of a network address for anetwork resource. For example, a computer system can not print adocument at a network printer unless the computer system (or anothernetworked computer system) knows the network address of the networkprinter.

Accordingly, various mechanisms (e.g., Domain Name System (“DNS”),Active Directory (“AD”), Distributed File Systems (“DFS”)) have beendeveloped for computer systems to identify (and access) previous unknownresources. However, due to the quantity and diversity of resources(e.g., devices and services) that are accessible via different computernetworks, developers are often required to develop applications thatimplement a variety of different resource identification and accessmechanisms. Each different mechanism may have different codingrequirements and may not provide a developer with all the functionalitythat is needed in an application.

For example, although DNS has a distributed administration architecture(i.e., centralized management is not required), DNS is not sufficientlydynamic, not self-organizing, supports a weak data and query model, andhas a fixed set of roots. On the other hand, AD is sufficiently dynamicbut requires centralized administration. Further, aspects of differentmechanisms may not be compatible with one another. For example, aresource identified using DNS may not be compatible with DFS routingprotocols. Thus, a developer may be forced to choose the most suitablemechanism and forgo the advantages of other mechanisms.

Mechanisms for identifying resources can be particularly problematic inpeer-to-peer networks. DNS provides a lookup service, with host names askeys and IP addresses as values, that relies on a set of special rootservers to implement lookup requests. Further, DNS requires managementof information (NS records) for allowing clients to navigate the nameserver hierarchy. Thus, a resource must be entered into DNS before theresource can be identified on a network. On larger scale networks wherenodes frequently connect and disconnect form the network relying onentry of information is not always practical. Additionally, DNS isspecialized to the task of find hosts or services and is not generallyapplicable to other types of resources.

Accordingly, other mechanisms for resource identification and accesshave been developed to attempt to address these shortcomings. A numberof mechanisms include distributed lookup protocols that are morescalable than DNS. These mechanisms use various node arrangements androuting algorithms to route requests to corresponding resources and tostore information for lookup.

At least one of these mechanisms utilizes local multi-level neighbormaps at each node in a network to route messages to a destination node.This essentially results in an architecture where each node is a “rootnode” of a corresponding tree of nodes (the nodes in its neighbor map).Messages are incrementally routed to a destination ID digit by digit(e.g., ***6=>**46=>, *346=>2346, where *s represent wildcards). Therouting efficiency of these types of mechanisms is O (log N) routinghops and require nodes to maintain a routing table of O (log N) size.

At least one other of these mechanisms assigns nodes a unique ID that istaken from a linear ring of numbers. Nodes maintain routing tables thatcontain pointers to their immediate successor node (according to IDvalue) and to those nodes whose ID values are the closest successor ofthe value ID+2^(L). The routing efficiency of these types of mechanismsis also O (log N) routing hops and require nodes to maintain a routingtable of O (log N) size.

At least one further mechanisms requires O (log N^(1/d)) routing hopsand requires nodes to maintain a routing table of O (D) size. Thus, therouting efficiency of all of these mechanisms depends, at least in part,on the number of nodes in the system.

Further, since IDs (for at least some of the mechanisms) can beuniformly distributed around a ring, there is always some possibilitythat routing between nodes on the ring will result in some inefficiency.For example, routing hops can cross vast geographic distances, crossmore expensive links, or pass through insecure domains, etc.Additionally, when message routing involves multiple hops, there is somechance that such events will occur multiple times. Unfortunately, thesemechanisms do not take into account the proximity of nodes (physical orotherwise) with respect one another. For example, depending on nodedistribution on a ring, routing a message from New York to Boston couldinvolve routing the message from New York, to London, to Atlanta, toTokyo, and then to Boston.

Accordingly, at least one other more recent mechanism takes proximityinto account by defining proximity as a single scalar proximity metric(e.g., IP routing hops or geographic distance). These mechanisms use thenotion of proximity-based choice of routing table entries. Since thereare many “correct” node candidates for each routing table entry, thesemechanisms attempt to select a proximally close node from among thecandidate nodes. For these mechanisms can provide a function that allowseach node to determine the “distance” of a node with a given IP addressto itself. Messages are routed between nodes in closer proximity to makeprogress towards a destination before routing to a node that is furtheraway. Thus, some resources can be conserved and routing is moreefficient.

Unfortunately, these existing mechanisms typically do not provide for,among other things, symmetric relationships between nodes (i.e., if afirst node considers a second node to be its partner, the second nodeconsiders the first node as a partner as well), routing messages in bothdirections (clockwise and counterclockwise) on a ring, partitioninglinked lists of nodes based on a plurality of proximity metrics, androuting messages based on a plurality of proximity metrics proximity.Therefore systems, methods, computer program products that utilize thesemechanisms to rendezvous resource requests with a corresponding resourcewould be advantageous.

BRIEF SUMMARY OF THE INVENTION

The foregoing problems with the prior state of the art are overcome bythe principles of the present invention, which are directed towardsmethods, systems, and computer program products for rendezvousingresource requests with corresponding resources. In some embodiments, thenodes of a federation infrastructure are partitioned. A sorted linkedlist containing node IDs that have been assigned to nodes in thefederation infrastructure is accessed. Proximity categories thatrepresent a plurality of different proximity criterion for partitioningthe sorted link list are accessed. The sorted linked list is partitionedinto one or more first sub lists based on a first proximity criterion,each of the one or more first sub lists containing at least a subset ofthe node IDs from the sorted linked list. A first sub list, selectedfrom among the one or more first sub lists, is partitioned in to one ormore second sub lists based on a second proximity criterion, each of theone or more second sub lists containing at least a subset of node IDscontained in the first sub list.

In other embodiments, for example as depicted in FIG. 3, a node routingtable is populated. An immediate predecessor node is inserted into therouting table. An immediate successor node is inserted into the routingtable. Appropriate neighborhood node identifiers are inserted into therouting table, the neighborhood nodes identified from the sorted linkedlist in both the first direction and in a second opposite directionbased on a predetermined or estimated neighborhood range andneighborhood size. Appropriate routing nodes identifiers are insertedinto the routing table, the routing nodes identified from the sortedlinked list in both the first and second directions based on the numberbase and field size of the ID space for the federation infrastructure,the routing nodes representing a logarithmic index of the sorted linklist in both the first and second directions.

In yet other embodiments, a node routing table can be populated takingproximity criteria in to account. A predecessor node for eachhierarchically partitioned routing ring the current node participates inis inserted into a routing table, each hierarchically partitionedrouting ring being partitioned in accordance with correspondingproximity criteria and containing at least subsets of the bi-directionallinked list of a parent ring. A successor node for each hierarchicallypartitioned routing ring the current node participates is inserted intothe routing table. Appropriate neighborhood nodes for eachhierarchically partitioned routing ring the current node participates inare inserted into the routing table. Appropriate routing nodes for eachhierarchically partitioned routing ring the current node participates inare inserted into the routing table.

In further other embodiments, a message is routed, potentially based onone or more proximity criteria defining a corresponding one or moreclasses of nodes, towards a destination node. A receiving node receivesa message along with a destination number indicating a destination nodeand optionally one or more proximity criteria. The receiving node,potentially among nodes in a current class of nodes, determines it is atleast one of numerically further from the destination number than acorresponding predecessor node and numerically further from thedestination number than a corresponding successor node. It is determinedthat the destination is not in a neighborhood set of nodes, potentiallyamong nodes in the current class of nodes, corresponding to thereceiving node.

An intermediate node from a routing table corresponding to the receivingnode is identified, the intermediate node being numerically closer tothe destination number than other routing nodes in the correspondingrouting table. The message is sent to the intermediate node. Theintermediate node can continue routing the message. The messageeventually reaches the destination node when a node that receives themessage is numerically closer to the destination number than either itssuccessor or predecessor nodes. In embodiments that route based on oneor more proximity criteria, this numerical closeness may be with respectto nodes in a selected class of nodes.

Thus, routing a message based on proximity criteria includes routing toa destination node (ID) by progressively moving closer to thedestination node within a given proximal ring (class of nodes) until nofurther progress can be made by routing within that ring. Determiningthat no further progress can be made occurs when the destination numberlies between the current node's ID and its successor or predecessornodes' IDs. At this point, the current node starts routing via itspartner nodes in the next larger proximal ring in which it participates.This process of progressively moving towards the destination node byclimbing along the partitioning path towards the root ring terminateswhen the destination node is reached.

These and other objects and features of the present invention willbecome more fully apparent from the following description and appendedclaims, or may be learned by the practice of the invention as set forthhereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

To further clarify the above and other advantages and features of thepresent invention, a more particular description of the invention willbe rendered by reference to specific embodiments thereof which areillustrated in the appended drawings. It is appreciated that thesedrawings depict only typical embodiments of the invention and aretherefore not to be considered limiting of its scope. The invention willbe described and explained with additional specificity and detailthrough the use of the accompanying drawings in which:

FIG. 1 illustrates an example of a federation infrastructure.

FIG. 2 illustrates an example of a computer architecture thatfacilitates routing request indirectly to partners.

FIG. 3 illustrates an example binary relationship between nodes in afederation infrastructure in the form of a sorted list and correspondingring.

FIG. 4 illustrates an example ring of rings that facilitates proximalrouting.

FIG. 5 illustrates an example proximity induced partition tree of ringsthat facilitates proximal routing.

FIG. 6 illustrates a suitable operating environment for the principlesof the present invention.

FIG. 7 illustrates an example flow chart of a method for populating anode routing table that takes proximity criteria into account

FIG. 8 illustrates an example flow chart of a method for partitioningthe nodes of a federation infrastructure.

FIG. 9 illustrates an example flow chart of a method for populating anode routing table.

FIG. 10 illustrates an example flow chart of a method for numericallyrouting a message towards a destination node.

FIG. 11 illustrates an example flow chart of a method for proximallyrouting a message towards a destination node.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The foregoing problems with the prior state of the art are overcome bythe principles of the present invention, which are directed towardsmethods, systems, and computer program products for rendezvousingresource requests with corresponding resources. In some embodiments, thenodes of a federation infrastructure are partitioned. A sorted linkedlist containing node IDs that have been assigned to nodes in thefederation infrastructure is accessed. Proximity categories thatrepresent a plurality of different proximity criterion for partitioningthe sorted link list are accessed. The sorted linked list is partitionedinto one or more first sub lists based on a first proximity criterion,each of the one or more first sub lists containing at least a subset ofthe node IDs from the sorted linked list. A first sub list, selectedfrom among the one or more first sub lists, is partitioned in to one ormore second sub lists based on a second proximity criterion, each of theone or more second sub lists containing at least a subset of node IDscontained in the first sub list.

In other embodiments, for example as depicted in FIG. 3, a node routingtable is populated. An immediate predecessor node is inserted into therouting table. An immediate successor node is inserted into the routingtable. Appropriate neighborhood node identifiers are inserted into therouting table, the neighborhood nodes identified from the sorted linkedlist in both the first direction and in a second opposite directionbased on a predetermined or estimated neighborhood range andneighborhood size. Appropriate routing nodes identifiers are insertedinto the routing table, the routing nodes identified from the sortedlinked list in both the first and second directions based on the numberbase and field size of the ID space for the federation infrastructure,the routing nodes representing a logarithmic index of the sorted linklist in both the first and second directions.

In yet other embodiments, a node routing table can be populated takingproximity criteria in to account. A predecessor node for eachhierarchically partitioned routing ring the current node participates inis inserted into a routing table, each hierarchically partitionedrouting ring being partitioned in accordance with correspondingproximity criteria and containing at least subsets of the bi-directionallinked list of a parent ring. A successor node for each hierarchicallypartitioned routing ring the current node participates is inserted intothe routing table. Appropriate neighborhood nodes for eachhierarchically partitioned routing ring the current node participates inare inserted into the routing table. Appropriate routing nodes for eachhierarchically partitioned routing ring the current node participates inare inserted into the routing table.

In further other embodiments, a message is routed, potentially based onone or more proximity criteria defining a corresponding one or moreclasses of nodes, towards a destination node. A receiving node receivesa message along with a destination number indicating a destination nodeand optionally one or more proximity criteria. The receiving node,potentially among nodes in a current class of nodes, determines it isnumerically further from the destination number than a correspondingpredecessor node and numerically further from the destination numberthan a corresponding successor node. It is determined that thedestination is not in a neighborhood set of nodes, potentially amongnodes in the current class of nodes, corresponding to the receivingnode.

An intermediate node from a routing table corresponding to the receivingnode is identified, the intermediate node being numerically closer tothe destination number than other routing nodes in the correspondingrouting table. The message is sent to the intermediate node. Theintermediate node can continue routing the message. The messageeventually reaches the destination node when a node that receives themessage is numerically closer to the destination number than either itssuccessor or predecessor nodes. In embodiments that route based on oneor more proximity criteria this numerical closeness may be with respectto nodes in a selected class of nodes.

Thus, routing a message based on proximity criteria includes routing toa destination node (ID) by progressively moving closer to thedestination node within a given proximal ring (class of nodes) until nofurther progress can be made by routing within that ring. Determiningthat no further progress can be made occurs when the destination numberlies between the current node's ID and its successor or predecessornodes' IDs. At this point, the current node starts routing via itspartner nodes in the next larger proximal ring in which it participates.This process of progressively moving towards the destination node byclimbing along the partitioning path towards the root ring terminateswhen the destination node is reached.

Embodiments within the scope of the present invention includecomputer-readable media for carrying or having computer-executableinstructions or data structures stored thereon. Such computer-readablemedia may be any available media, which is accessible by ageneral-purpose or special-purpose computer system. By way of example,and not limitation, such computer-readable media can comprise physicalstorage media such as RAM, ROM, EPROM, CD-ROM or other optical diskstorage, magnetic disk storage or other magnetic storage devices, or anyother media which can be used to carry or store desired program codemeans in the form of computer-executable instructions, computer-readableinstructions, or data structures and which may be accessed by ageneral-purpose or special-purpose computer system.

In this description and in the following claims, a “network” is definedas one or more data links (of possibly different speeds) that enable thetransport of electronic data between computer systems and/or modules(e.g., hardware and/or software modules). When information istransferred or provided over a network or another communicationsconnection (either hardwired, wireless, or a combination of hardwired orwireless) to a computer system, the connection is properly viewed as acomputer-readable medium. Thus, any such connection is properly termed acomputer-readable medium. Combinations of the above should also beincluded within the scope of computer-readable media.Computer-executable instructions comprise, for example, instructions anddata which cause a general-purpose computer system or special-purposecomputer system to perform a certain function or group of functions. Thecomputer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, or evensource code. In some embodiments, hardware modules, such as, forexample, special purpose integrated circuits or Gate-arrays areoptimized to implement the principles of the present invention.

In this description and in the following claims, a “node” is defined asone or more software modules, one or more hardware modules, orcombinations thereof, that work together to perform operations onelectronic data. For example, the definition of a node includes thehardware components of a personal computer, as well as software modules,such as the operating system of the personal computer. The physicallayout of the modules is not important. A node can include one or morecomputers coupled via a network. Likewise, a node can include a singlephysical device (such as a mobile phone or Personal Digital Assistant“PDA”) where internal modules (such as a memory and processor) worktogether to perform operations on electronic data. Further, a node caninclude special purpose hardware, such as, for example, a router thatincludes special purpose integrated circuits.

Those skilled in the art will appreciate that the invention may bepracticed in network computing environments with many types of nodeconfigurations, including, personal computers, laptop computers,hand-held devices, multi-processor systems, microprocessor-based orprogrammable consumer electronics, network PCs, minicomputers, mainframecomputers, mobile telephones, PDAs, pagers, routers, gateways, brokers,proxies, firewalls, redirectors, network address translators, and thelike. The invention may also be practiced in distributed systemenvironments where local and remote nodes, which are linked (either byhardwired data links, wireless data links, or by a combination ofhardwired and wireless data links) through a network, both performtasks. In a distributed system environment, program modules may belocated in both local and remote memory storage devices.

Federation Architecture

FIG. 1 illustrates an example of a federation infrastructure 100. Thefederation infrastructure 100 includes nodes 101, 102, 103, that canform different types of federating partnerships. For example, nodes 101,102, 103 can be federated among one another as peers without a rootnode. Each of nodes 101, 102, and 103 has a corresponding ID 171, 182,and 193 respectively.

Generally, the nodes 101, 102, 103, can utilize federation protocols toform partnerships and exchange information (e.g., state informationrelated to interactions with other nodes). The formation of partnershipsand exchange of information facilitates more efficient and reliableaccess to resources. Other intermediary nodes (not shown) can existbetween nodes 101, 102, and 103 (e.g., nodes having IDs between 171 and193). Thus, a message routed, for example, between node 101 and node103, can be pass through one or more of the other intermediary nodes.

Nodes in federation infrastructure 100 (including other intermediarynodes) can include corresponding rendezvous protocol stacks. Forexample, nodes 101, 102, and 103 include corresponding rendezvousprotocol stacks 141, 142, and 143 respectively. Each of the protocolsstacks 141, 142, and 143 includes an application layer (e.g.,application layers 121, 122, and 123) and other lower layers (e.g.,corresponding other lower layers 131, 132, and 133). Each layer in arendezvous protocol stack is responsible for different functionalityrelated to rendezvousing a resource request with a correspondingresource.

For example, other lower layers can include a channel layer, a routinglayer, and a function layer. Generally, a channel layer is responsiblefor reliably transporting a message (e.g., using WS-ReliableMessagingand Simple Object Access Protocol (“SOAP”)) from one endpoint to another(e.g., from node 101 to node 103). The channel layer is also responsiblefor processing incoming and outgoing reliable messaging headers andmaintaining state related to reliable messaging sessions.

Generally, a routing layer is responsible for computing the next hoptowards a destination. The routing layer is also responsible forprocessing incoming and outgoing addressing and routing message headersand maintaining routing state. Generally, a function layer isresponsible for issuing and processing rendezvous protocol messages suchas join and depart requests, pings, updates, and other messages, as wellas generation of responses to these messages. The function layerprocesses request messages from the routing layer and sends backcorresponding response messages, if any, to the originating node usingthe routing layer. The function layer also initiates request messagesand utilizes the routing layer to have the requests messages delivered.

Generally, an application layer processes non-rendezvous protocolspecific data delivered from the function layer (i.e., applicationmessages). The function layer can access application data from theapplication layer and get and put application data in rendezvousprotocol messages (e.g., pings and updates). That is, the function layercan cause application data to be piggybacked on rendezvous protocolmessages and can cause the application data to be passed back to theapplication layer in receiving rendezvous protocol nodes. In someembodiments, application data is used to identify resources and resourceinterests. Thus, an application layer can include application specificlogic and state that processes data received from and sent to the otherlower layers for purposes of identifying resources and resourceinterests.

Federating Mechanisms

Nodes can federate using a variety of different mechanisms. A firstfederating mechanism includes peer nodes forwarding information to allother peer nodes. When a node is to join a federation infrastructure,the node utilizes a broadcast/multicast discovery protocol, such as, forexample, WS-Discovery to announce its presence and issues abroadcast/multicast find to detect other nodes. The node thenestablishes a simple forwarding partnership with other nodes alreadypresent on the network and accepts new partnerships with newly joiningnodes. Thereafter, the node simply forwards all application specificmessages to all of its partner nodes.

A second federating mechanism includes peer nodes that most efficientlytransmit application specific messages to their destination(s). When anew node is to join a federation infrastructure, the new node utilizes abroadcast/multicast discovery protocol, such as, for example,WS-Discovery to announce its presence and issues a broadcast/multicastfind to detect other nodes that are part of the federationinfrastructure. Upon detecting another node, the new node establishes apartnership with the other node. From the established partnership, thenew node learns about the presence of other nodes already participatingin federation infrastructure. It then establishes partnerships withthese newly-learned nodes and accepts any new incoming partnershiprequests.

Both node arrivals/departures and registrations of interest in certainapplication specific messages are flooded through the federationinfrastructure resulting in every node having global knowledge of otherpartner nodes and registrations of interest in application specificmessages. With such global knowledge, any node can send applicationspecific messages directly to the nodes that have expressed interest inthe application specific message.

A third federating mechanism includes peer nodes indirectly forwardingall application specific messages to their destination/s. In this thirdmechanism, nodes are assigned identifiers (ID's), such as, for example,a 128-bit or 160-bit ID. The node responsible for a maintainingregistration of interest in a given application specific message can bedetermined to be the one whose ID is closest to the one obtained bymapping (e.g., hashing) the destination identity (e.g. URI) of theapplication specific message to this 128-bit or 160-bit ID-space.

In this third mechanism, node arrivals and departures are flooded overthe entire fabric. On the other hand, registrations of interest incertain application specific messages are forwarded to the nodesdetermined to be responsible for maintaining such registrationinformation. For scalability, load balancing, and fault-tolerance, thenode receiving registration of interest in certain application specificmessages can reliably flood that registration information within itsneighborhood set. The neighborhood set for a specified node can bedetermined to be the set of nodes having IDs within a predefined rangeon either side of the ID of specified node.

Similar to the second mechanism, a newly-joining node utilizes abroadcast/multicast discovery protocol, such as, for example,WS-Discovery to announce its presence and issues a localbroadcast/multi-cast find to detect a node that is already part of thefederation infrastructure. The new node establishes a partnership withthe discovered node and uses that partnership to learn about thepresence of other new nodes participating in the federationinfrastructure. The new node then establishes further partnerships withthe newly discovered nodes and accepts any new incoming partnershiprequests. The new node accepts incoming registrations of interest incertain application layer specific resources from its partners for whichit is responsible and may flood them over its neighborhood set. Thus,messages can generally be forwarded to their final destination viaintermediary routing nodes (e.g., that a newly joining node haspartnered with or that a partner node is aware of).

In response to receiving an incoming application specific message, thenew node forwards the message to the partner node that may beresponsible for maintaining the registration information for thedestination specified in the message. Thus, when using this thirdmechanism, every node in the federation infrastructure has globalknowledge of all other nodes but the registration information isefficiently partitioned among the nodes. Application specific messagesare transmitted to their final destination via only the partner's nodesthat may have the responsibility for maintaining registrationinformation of interest in those application specific messages. Thus,indirection is accomplished by forwarding only to the partner node thathas global knowledge of the registration information of interest for themessage being processed. This is in contrast to the first mechanismwhere the indirection is accomplished by forwarding to all the partnernodes.

A fourth federating mechanism includes peer nodes that route messages toother peer nodes. This fourth mechanism differs from the third mechanismat least in that both node arrivals/departures and registrations ofinterest in certain application specific messages are all routed insteadbeing flooded. Routing protocols are designed to guarantee rendezvousbetween application specific messages and the registration messages thatexpress interest in those application specific messages.

FIG. 2 illustrates an example of a computer architecture 200 thatfacilitates routing requests indirectly to partners. Computerarchitecture 200 depicts different types of computer systems and devicespotentially spread across multiple local discovery scopes participatingin a federation infrastructure.

Workstation 233 can include a registered PnP provider instance. Toinform its partners of the presence of this PnP provider instance,workstation 233 routes registration request 201 over the federationinfrastructure. Registration request 201 is initially forwarded tolaptop 231, which in turn forwards registration request 201 to messagebroker 237, which in turn forwards registration request 201 to messagegateway 241. Message gateway 241 saves the registration informationregistration request 201 in its database and returns success message 204to workstation 233.

Subsequently, another registered provider instance, this time that ofrunning services, comes alive within the workstation 233. This time thenode is aware that message gateway 241 is responsible for registrationsand forwards registration request 205 to message gateway 241 directly.Message gateway 241 saves the registration information registrationrequest 205 in its database and returns success message 206 toworkstation 233.

Subsequently, the printer 236 (e.g., a UPnP printer) is powered on andsends announcement 207. Server 234 detects announcement 207 and routesregistration request 208 to message broker 237. Message broker 237forwards registration request 208 to message gateway 241. Messagegateway 241 saves the registration information registration request 208in its database and returns success message 210 to server 234.

Subsequently, personal computer 242 issues lookup request 211 todiscover all devices. Since personal computer 242 doesn't know where toforward lookup request 211, it routes lookup request 211 throughworkstation 243. As registration and lookup requests are routed to thesame destination, the routing protocol essentially guarantees rendezvousbetween the two requests resulting in workstation 243 forwards findrequest 211 to message gateway 241. Message gateway 241 looks up theregistration information maintained by it and forwards find request 211to both the workstation 233 and server 234. Workstation 233 and server234 send response messages 214 and 216 respectively to personal computer242.

This fourth mechanism works by routing (instead of flooding) a requestto the node (message gateway 241) that has global knowledge of theregistrations specified in a request. This fourth mechanism, as will bedescribed in further detail below, essentially guarantees that routingcan be accomplished in O (log N) hops, where N is the number of nodesparticipating in the federation infrastructure. Since this fourthmechanism efficiently partitions both node partnership and registrationinformation, it scales to very large networks, even the Internet.

Although a number of federating mechanisms have been described, it wouldbe apparent to one skilled in the art, after having reviewed thisdescription, that other federation mechanisms are possible.

Relationship Between Nodes in a Federation

Accordingly, a federation consists of a set of nodes that cooperateamong themselves to form a dynamic and scalable network in whichinformation can be systematically and efficiently disseminated andlocated. Nodes are organized to participate in a federation as a sortedlist using a binary relation that is reflexive, anti-symmetric,transitive, total, and defined over the domain of node identities. Bothends of the sorted list are joined, thereby forming a ring. Thus, eachnode in the list can view itself as being at the middle of the sortedlist (as a result of using modulo arithmetic). Further, the list isdoubly linked so that any node can traverse the list in eitherdirection.

Each federating node can be assigned an ID (e.g., by a random numbergenerator with duplicate detection) from a fixed set of IDs between 0and some fixed upper bound. Thus, adding 1 to an ID of the fixed upperbound results in an ID of zero (i.e., moving from the end of the linkedlist back to the beginning of the linked listed . In addition, a 1:1mapping function from the value domain of the node identities to thenodes themselves is defined.

FIG. 3 depicts an example linked list 304 and corresponding ring 306.Given such a ring, the following functions can be defined:

-   RouteNumerically (V, Msg): Given a value V from the value domain of    node identities and a message “Msg,” deliver the message to node X    whose identity can be mapped to V using the mapping function.-   Neighborhood (X, S): Neighborhood is the set of nodes on the either    side of node X with cardinality equal to S.

When every node in the federation has global knowledge of the ring,RouteNumerically (V, Msg) is implemented by directly sending Msg to thenode X, whose identity is obtained by applying the mapping function toV. Alternately, when nodes have limited knowledge of other nodes (e.g.,only of immediately adjacent nodes), RouteNumerically (V, Msg) isimplemented by forwarding the message to consecutive nodes along thering until it reaches the destination node X.

Alternately (and advantageously), nodes can store enough knowledge aboutthe ring to perform a distributed binary search (without having to haveglobal knowledge or implement routing between immediately adjacentnodes). The amount of ring knowledge is configurable such thatmaintaining the ring knowledge has a sufficiently small impact on eachnode but allows increased routing performance from the reduction in thenumber of routing hops.

As previously described, IDs can be assigned using the “<” (less than)relation defined over a sufficiently large, bounded set of naturalnumbers, meaning its range is over a finite set of numbers between 0 andsome fixed value, inclusive. Thus, every node participating in thefederation is assigned a natural number that lies between 0 and someappropriately-chosen upper bound, inclusive. The range does not have tobe tight and there can be gaps between numbers assigned to nodes. Thenumber assigned to a node serves as its identity in the ring. Themapping function accounts for gaps in the number space by mapping anumber falling in between two node identities to the node whose identityis numerically closest to the number.

This approach has a number of advantages. By assigning each node auniformly-distributed number, there is an increased likelihood that allsegments of the ring are uniformly populated. Further, successor,predecessor, and neighborhood computations can be done efficiently usingmodulo arithmetic.

In some embodiments, federating nodes are assigned an ID from within anID space so large that the chances of two nodes being assigned the sameID are highly unlikely (e.g., when random number generation is used).For example, a node can be assigned an ID in the range of 0 to b^(n)−1,where b equals, for example, 8 or 16 and n equals, for example, 128-bitor 160-bit equivalent digits. Accordingly, a node can be assigned an ID,for example, from a range of 0 to 16⁴⁰−1 (or approximately 1.461502E48).The range of 0 to 16⁴⁰−1 would provide, for example, a sufficient numberof IDs to assign every node on the Internet a unique ID.

Thus, each node in a federation can have:

-   An ID which is a numerical value uniformly distributed in the range    of 0 to b^(n)−1; and-   A routing table consisting of (all arithmetic is done modulo b^(n)):    -   Successor node (s);    -   Predecessor node (p);    -   Neighborhood nodes (p_(k), . . . , p₁, p, s, s₁, . . . , s_(j))        such that s_(j).s.id>(id+u/2), j≧v/2−1, and p_(k).p.id<(id−u/2),        and k≧v/2−1; and    -   Routing nodes (r_(-(n−1)), . . . , r⁻¹, r₁, . . . , r_(n−1))        such that r_(±i)=RouteNumerically (id±b^(i), Msg).        where b is the number base, n is the field size in number of        digits, u is the neighborhood range, v is the neighborhood size,        and the arithmetic is performed modulo b^(n). For good routing        efficiency and fault tolerance, values for u and v can be u=b        and v≧max(log₂(N), 4), where N is the total number of nodes        physically participating in the federation. N can be estimated        from the number of nodes present on a ring segment whose length        is greater than or equal to b, for example, when there is a        uniform distribution of IDs. Typical values for b and n are b=8        or 16 and n=128-bit or 160-bit equivalent digits.

Accordingly, routing nodes can form a logarithmic index spanning a ring.Depending on the locations of nodes on a ring, a precise logarithmicindex is possible, for example, when there is an existing node at eachnumber in the set of id±b^(i) where i=(1, 2, . . . (n−1)). However, itmay be that there are not existing nodes at each number in the set. INthose cases, a node closest to id±b^(i) can be selected as a routingnode. The resulting logarithmic index is not precise and may even lackunique routing nodes for some numbers in the set.

Referring again to FIG. 3, FIG. 3 illustrates an example of a binaryrelation between nodes in a federation infrastructure in the form ofsorted list 304 and corresponding ring 306. The ID space of sorted list304 is in the range 0 to 2⁸−1 (or 255). That is, b=2 and n=8. Thus,nodes depicted in FIG. 3 are assigned IDs in a range from 0 to 255.Sorted list 304 utilizes a binary relation that is reflexive,anti-symmetric, transitive, total, and defined over the domain of nodeidentities. Both ends of sorted list 304 are joined, thereby formingring 306. This makes it possible for each node in FIG. 3 to view itselfas being at the middle of sorted list 304. The sorted list 304 is doublylinked so that any node can traverse the sorted list 304 in eitherdirection. Arithmetic for traversing sorted list 304 (or ring 306) isperformed modulo 2⁸. Thus, 255 (or the end of sorted list 304)+1=0 (orthe beginning of sorted list 304).

The routing table indicates that the successor to ID 64 is ID 76 (the IDimmediately clockwise from ID 64). The successor can change, forexample, when a new node (e.g., with an ID of 71) joins or an existingnode (e.g., ID 76) leaves the federation infrastructure. Likewise, therouting table indicates that the predecessor to ID 64 is ID 50 (the IDimmediately counters clockwise from ID 64). The predecessor can change,for example, when a new node (e.g., with an ID of 59) joins or anexisting node (e.g., ID 50) leaves the federation infrastructure.

The routing table further indicates that a set of neighborhood nodes toID 64 have IDs 83, 76, 50 and 46. A set of neighbor nodes can be aspecified number of nodes (i.e., neighborhood size v) that are within aspecified range (i.e., neighbor range u) of ID 64. A variety ofdifferent neighborhood sizes and neighbor ranges, such as, for example,V=4 and U=10, can potentially be used to identify the set ofneighborhood nodes. A neighborhood set can change, for example, whennodes join or leave the federation infrastructure or when the specifiednumber of nodes or specified range is changed.

The routing table further indicates that ID 64 can route to nodes havingIDs 200, 2, 30, 46, 50, 64, 64, 64, 64, 76, 83, 98, 135, and 200. Thislist is generated by identifying the node closest to each number in theset of id±2^(i) where i=(1, 2, 3, 4, 5, 6, 7). That is, b=2 and n=8. Forexample, the node having ID 76 can be identified from calculating theclosest node to 64+2³, or 72.

A node can route messages (e.g., requests for access to resources)directly to a predecessor node, a successor node, any node in a set ofneighborhood nodes, or any routing node. In some embodiments, nodesimplement a numeric routing function to route messages. Thus,RouteNumerically (V, Msg) can be implemented at node X to deliver Msg tothe node Y in the federation whose ID is numerically closest to V, andreturn node Y's ID to node X. For example, the node having ID 64 canimplement RouteNumerically (243, Msg) to cause a message to be routed tothe node having ID 250. However, since ID 250 is not a routing node forID 64, ID 64 can route the message to ID 2 (the closest routing node to243). The node having ID 2 can in turn implement RouteNumerically (243,Msg) to cause the message to be routed (directly or through furtherintermediary nodes) to the node having ID 250. Thus, it may be that aRouteNumerically function is recursively invoked with each invocationrouting a message closer to the destination.

Advantageously, other embodiments of the present invention facilitatepartitioning a ring into a ring of rings or tree of rings based on aplurality of proximity criteria of one or more proximity categories(e.g., geographical boundaries, routing characteristics (e.g., IProuting hops), administrative domains, organizational boundaries, etc.).It should be understood a ring can be partitioned more than once usingthe same type of proximity criteria. For example, a ring can bepartition based on a continent proximity criteria and a countryproximity criteria (both of a geographical boundaries proximitycategory).

Since IDs can be uniformly distributed across an ID space (a result ofrandom number generation) there is a high probability that any givensegment of a circular ID space contains nodes that belong to differentproximity classes provided those classes have approximately the samecardinality. The probability increases further when there are asufficient number of nodes to obtain meaningful statistical behavior.

Thus, neighborhood nodes of any given node are typically well dispersedfrom the proximality point of view. Since published application statecan be replicated among neighborhood nodes, the published informationcan be well dispersed as well from the proximality point of view.

FIG. 4 illustrates a ring of rings 400 that facilitates proximalrouting. Ring 401 can be viewed as a master or root ring, and containsall the nodes in each of the rings 402, 403, and 404. Each of the rings402, 403, and 404 contain a subset of nodes from ring 401 that arepartitioned based on a specified proximity criterion. For example, ring401 may be partitioned based on geographic location, where ring 402contains nodes in North America, ring 403 contains nodes in Europe, andring 404 contains nodes in Asia.

In a numerical space containing 65,536 (2¹⁶) IDs, routing a message froma North American node having an ID 5,345 to an Asian node having an ID23,345 can include routing the message within ring 402 until a neighbornode of the Asian node is identified. The neighbor node can then routethe message to the Asian node. Thus, a single hop (as opposed tomultiple hops) is made between a North American node and an Asian node.Accordingly, routing is performed in a resource efficient manner.

FIG. 5 illustrates an example proximity induced partition tree of rings500 that facilitates proximal routing. As depicted, partition tree ofrings 500 includes a number of rings. Each of the rings represents apartition of a sorted linked list. Each ring including a plurality anodes having IDs in the sorted linked list. However for clarity due tothe number of potential nodes, the nodes are not expressly depicted onthe rings (e.g., the ID space of partition tree 500 may be b=16 andn=40).

Within partition tree 500, root ring 501 is partitioned into a pluralityof sub-rings, including sub-rings 511, 512, 513, and 514, based oncriterion 571 (a first administrative domain boundary criterion). Forexample, each component of a DNS name can be considered a proximitycriterion with the partial order among them induced per their order ofappearance in the DNS name read right to left. Accordingly, sub-ring 511can be further partitioned into a plurality of sub-rings, includingsub-rings 521, 522, and 523, based on criterion 581 (a secondadministrative domain boundary criterion).

Sub-ring 522 can be further partitioned into a plurality of sub-rings,including sub-rings 531, 532, and 533, based on criterion 572 (ageographic boundary criterion). Location based proximity criterion canbe partially ordered along the lines of continents, countries, postalzip codes, and so on. Postal zip codes are themselves hierarchicallyorganized meaning that they can be seen as further inducing a partiallyordered sub-list of proximity criteria.

Sub-ring 531 can be further partitioned into a plurality of sub-rings,including sub-rings 541, 542, 543, and 544, based on criterion 573 (afirst organizational boundary criterion). A partially ordered list ofproximity criterion can be induced along the lines of how a givencompany is organizationally structured such as divisions, departments,and product groups. Accordingly, sub-ring 543 can be further partitionedinto a plurality of sub-rings, including sub-rings 551 and 552, based oncriterion 583 (a second organizational boundary criterion).

Within partition tree 500, each node has a single ID and participates inrings along a corresponding partition path starting from the root to aleaf. For example, each node participating in sub-ring 552 would alsoparticipate in sub-rings 543, 531, 522, 511 and in root 501. Routing toa destination node (ID) can be accomplished by implementing aRouteProximally function, as follows:

-   RouteProximally (V, Msg, P): Given a value V from the domain of node    identities and a message “Msg,” deliver the message to the node Y    whose identity can be mapped to V among the nodes considered    equivalent by the proximity criteria P.

Thus, routing can be accomplished by progressively moving closer to thedestination node within a given ring until no further progress can bemade by routing within that ring as determined from the condition thatthe destination node lies between the current node and its successor orpredecessor node. At this point, the current node starts routing via itspartner nodes in the next larger ring in which it participates. Thisprocess of progressively moving towards the destination node by climbingalong the partitioning path towards the root ring terminates when theclosest node to the destination node is reached within the requestedproximal context, as originally specified in the RouteProximallyinvocation.

Routing hops can remain in the proximal neighborhood of the node thatoriginated the request until no further progress can be made within thatneighborhood because the destination node exists outside it. At thispoint, the proximity criterion is relaxed to increase the size of theproximal neighborhood to make further progress. This process is repeateduntil the proximal neighborhood is sufficiently expanded to include thedestination node (ID). The routing hop made after each successiverelaxation of proximal neighborhood criterion can be a potentiallylarger jump in proximal space while making a correspondingly smallerjump in the numerical space compared to the previous hop. Thus, only theabsolutely required number of such (inter-ring) hops is made before thedestination is reached.

It may be the case that some hops are avoided for lookup messages sincepublished application data gets replicated down the partition tree whenit is replicated among the neighborhood nodes of the destination node.

To accomplish proximal routing, each federation node maintainsreferences to its successor and predecessor nodes in all the rings itparticipates as a member (similar to successor and predecessor for asingle ring)—the proximal predecessor, proximal successor, and proximalneighborhood. In order to make the routing efficient, the nodes can alsomaintain reference to other nodes closest to an exponentially increasingdistance on its either half of the ring as routing partners (similar torouting nodes for a single ring). In some embodiments, routing partnernodes that lie between a pair of consecutive successor or predecessornodes participate in the same lowest ring shared by the current node andthe node numerically closest to it among the successor or predecessornode pairs respectively. Thus, routing hops towards a destination nodetransition into using a relaxed proximity criterion (i.e., transitioningto a higher ring) only when absolutely needed to make further progress.Accordingly, messages can be efficiently rendezvoused with acorresponding federation node.

In some embodiments, nodes implement a proximal routing function toroute messages based on equivalence criteria relations. Thus, given anumber V and a message “Msg”, a node can implement RouteProximally (V,Msg, P) to deliver the message to the node Y whose identify can bemapped to V among the nodes considered equivalent by proximity criterionP. The proximity criterion P identifies the lowest ring in the partitiontree that is the common ancestor to all the nodes considered proximallyequivalent by it. It can be represented as a string obtained byconcatenating the proximity criterion found along the path from the rootring to the ring identified by it separated by the path separatorcharacter ‘/’. For example, the proximity criterion identifying sub-ring542 can be represented as “Proximity:/.COM/Corp2/LocationA/Div2”. Eachring in the partition tree 500 can be assigned a unique number, forexample, by hashing its representational string with a SHA basedalgorithm. If the number 0 is reserved for the root ring, it can beinferred that RouteNumerically (V, Msg)≡RouteProximally (V, Msg, 0).

For example, a node in sub-ring 544 can implement RouteProximally toidentify a closer node in sub-ring 531 (e.g., to a node in sub-ring513). In turn, sub-ring 531 can implement RouteProximally to identify acloser node in sub-ring 522. Likewise, sub-ring 522 can implementRouteProximally to identify a closer node in sub-ring 511. Similarly,sub-ring 511 can implement RouteProximally to identify a closer node inring 501. Thus, it may be that a RouteProximally function is recursivelyinvoked with each invocation routing a message closer to thedestination.

Thus, when proximity criterion is taken into account, routing hops on apath to a final destination can remain within the proximity of a nodethat originates a request, while making significant progress between theoriginating node and the destination node in a numerical space, untileither the destination node is reached or no further progress can bemade under the chosen proximity criterion at which point it is relaxedjust enough to make further progress towards the destination. Forexample, proximity criterion can be relaxed enough for a message to berouted from ring 531 up to ring 522, etc.

Utilizing the above approach to proximity, it is possible to confinepublished information to a given ring. For example, organizations maylike to ensure that organization specific information is not availableto entities outside of their trust domains either (1) implicitly in theform of neighborhood replication to nodes outside of their domains or(2) explicitly in the form of servicing lookup requests for suchinformation. The first aspect is satisfied by replicating publishedinformation only among the nodes neighboring the target ID within thespecified ring. Because all messages originated by a node are routed bysuccessively climbing the rings to which it belongs towards the rootring, there is a high likelihood that all lookup requests originatedwithin an organization will be able to locate the published informationconfined to it thereby implicitly satisfying the second aspect.

Also, organizations dislike nodes automatically federating with nodesoutside of their trust domain. This can happen, for example, when avisiting sales person connects his/her laptop computer to the network inthe customer premises. Ideally, the laptop computer belonging to thesales person wishes to locate information published in its home domainand/or federate with the nodes in its home domain starting at its lowestpreferred proximity ring. It will typically not be permitted to federatewith the nodes in the customer's domain. Supporting this scenariorequires ability to locate seed nodes in the home domain. Such seednodes can be used for locating information published in the home domain,to join the home federation, and selectively import and export publishedinformation across domains. Seed nodes are also sometimes referred asmessage gateways.

In other embodiments, an entity publishes references to seed nodes inthe root ring. Seed nodes can be published at the unique number (such asthe one obtained by hashing its representational string) associated withthe ring (as a target ID). Seed node information can further beon-demand cached by the nodes in various rings that are on the path tothe corresponding target IDs in the root ring. Such on-demand cachingprovides for improved performance and reduction in hotspots that mightoccur when semi-static information is looked up quite frequently. Seednode information can also be obtained via other means such as DNS

To provide fault tolerance for confined published information, each nodecan maintain a set of neighborhood nodes in all of the rings itparticipates in. Given the above, the state maintained by a node can besummarized as follows:

-   An ID which is a numerical value uniformly distributed in the range    of 0 to b^(n)−1.-   A routing table consisting of (all arithmetic is done modulo b^(n)):    -   For each ring, say ring d, in which the node participates        -   Successor node (s_(d))        -   Predecessor node (p_(d))        -   Neighborhood nodes (p_(kd), . . . , p_(1d), p_(d), s_(d),            s_(1d), . . . , s_(jd)) such that s_(jd).s_(d).id>(id+u/2),            j≧v/2−1, p_(kd).p_(d).id<(id−u/2), and k≧v/2−1.    -   Routing nodes (r_(−(n−1)), . . . , r⁻¹, r₁, . . . , r_(n−1))        such that r_(±i)=RouteProximally (id±b^(i), updateMsg, d) such        that s_(d)≦id+b^(i)≦s_(d+1) or p_(d+1)≦id−b^(i)≦p_(d) as        appropriate.        where b is the number base, n is the field size in number of        digits, u is the neighborhood range, and v is the neighborhood        size.

Note that a subset of the neighborhood nodes maintained by a given nodein ring “d” can appear again as neighborhood nodes in the child ring“d+1” in which the given node participates as well. As such one canderive the upper bound on the total number of neighborhood nodesmaintained by a given node across all the D rings it participates asD*max(u,v)/2. This considers that only one reference to a given node iskept and the worst case upper bound is for a balanced tree.

It should be noted that when a ring is partitioned into a plurality ofcorresponding sibling sub-rings, it is permitted for a specified node tosimultaneously participate in more than one of the plurality ofcorresponding sibling sub-rings, for example, through aliasing. Aliasingcan be implemented to associate different state, for example, fromdifferent sub-rings, with the specified node. Thus, although aliases fora given node have the same ID, each alias can have distinct stateassociated with them. Aliasing allows the specified node to participatein multiple rings having distinct proximity criteria that are notnecessarily common ancestors of more specific proximity criteria. Thatis, the specified node can participate in multiple branches of theproximity tree.

For example, a dual NIC (wired and wireless) laptop can be considered tobe proximally equivalent to both other wireless and wired nodes sharingthe same LAN segments as the laptop. But, these two distinct proximitycriteria can be modeled as sub-criteria that are applicable only afterapplication of a different higher priority proximity criterion, such as,for example, one based on organizational membership. As the laptopbelongs to the same organization, the aliased nodes in the two sub-ringsrepresenting 1) membership in the wired and 2) membership in thewireless LAN segments merge into a single node in the ring representingthe organization to which the laptop belongs. It should be understandthat the RouteProximally works as expected without any modifications inthe presence of aliasing.

Each proximal ring can be configured in accordance with (potentiallydifferent) ring parameters. Ring parameters can be used to define aneighborhood (e.g., ring parameters can represent a neighborhood range,a neighborhood size, ping message and depart message timing anddistribution patterns for ping and depart messages), indicate aparticular federating mechanisms (e.g., from among the above-describedfirst through fourth federating mechanisms previously described or fromamong other federating mechanisms), or define communication specificsbetween routing partners in the same proximal ring. Some ring parametersmay be more general, applying to a plurality of different federatingmechanisms, while other ring parameters are more specific and apply tospecific type of federating mechanism.

Ring parameters used to configure a higher level proximal ring can beinherited in some embodiments by lower level proximal rings. Forexample, it may be that ring 543 inherits some of the ring parameters ofring 531 (which in turn inherited from ring 522, etc.). Thus, aneighborhood size and neighborhood range associated with ring 531 isalso associated with ring 541.

However, inherited ring parameters can be altered and/or proximal ringscan be individually configured in accordance with different ringparameters. For example, it may be that ring 511 is for anadministrative domain that contains a large number of nodes and thus theabove-described fourth federating mechanism is more appropriate for ring511. On the other hand, it may be that ring 521 is for a small businesswith a relatively smaller number of nodes and thus the above-describedsecond federating mechanism is more appropriate for ring 521. Thus, thering parameters associated with ring 521 can be set to (or inheritedparameters changed to) different values than the ring parametersassociated with ring 511. For example, a ring parameter indicating aparticular type of federating mechanisms can be different between rings511 and 521. Similarly parameters defining a neighborhood can bedifferent between rings 511 and 521. Further, ring 521 can be configuredin accordance with specific parameters that are specific to theabove-described second federating mechanism, while ring 511 isconfigured in accordance additional with specific parameters that arespecific to the above-described fourth federating mechanism.

Accordingly, proximal rings can be flexibly configured based on thecharacteristics (e.g., number, included resources, etc.) of nodes in theproximal rings. For example, an administrator can select ring parametersfor proximal rings using a configuration procedure (e.g., through auser-interface). A configuration procedure can facilitate theconfiguration of inheritance relationships between proximal rings aswell as the configuration of individual proximal rings, such as, forexample, to override otherwise inherited ring parameters.

FIG. 8 illustrates an example flow chart of a method 800 forpartitioning the nodes of a federation infrastructure. The method 800will be described with respect to the rings of partition a tree 500 inFIG. 5. Method 800 includes an act of accessing a sorted linked listcontaining node IDs that have been assigned to nodes in a federationinfrastructure (act 801). For example, the sorted linked listrepresented by ring 501 can be accessed. The node IDs of the sortedlinked list (the nodes depicted on ring 501) can represent nodes in afederation infrastructure (e.g., federation infrastructure 100).

Method 800 includes an act of accessing proximity categories thatrepresent a plurality of different proximity criteria for partitioningthe sorted linked list (act 802). For example, proximity criterionrepresenting domain boundaries 561, geographical boundaries 562, andorganizational boundaries 563 can be accessed. However, other proximitycriteria, such as, trust domain boundaries, can also be represented inaccessed proximity criterion. Proximity categories can includepreviously created partially ordered lists of proximity criteria. A ringcan be partitioned based on partially ordered lists of proximitycriteria.

Method 800 includes an act of partitioning the sorted link list into oneor more first sub lists based on a first proximity criterion, each ofthe one or more first sub lists containing at least a subset of the nodeIDs from the sorted linked list (act 803). For example, ring 501 can bepartitioned into sub-rings 511, 512, 513, and 514 based on criterion571. Each of sub-rings 511, 512, 513, and 514 can contain a differentsub-set of node IDs from ring 501.

Method 800 includes an act of partitioning a first sub list, selectedfrom among the one or more first sub lists, into one or more second sublists based on a second proximity criterion, each of the one or moresecond sub lists containing at least a subset of node IDs contained inthe first sub list (act 804). For example, sub-ring 511 can bepartitioned into sub-rings 521, 522, and 523 based on criterion 581.Each of the sub-rings 521, 522, and 523 can contain a different sub-setof node IDs from sub-ring 511.

FIG. 9 illustrates an example flow chart of a method 900 for populatinga node's routing table. The method 900 will be described with respect tothe sorted linked list 304 and ring 306 in FIG. 3. Method 900 includesan act of inserting a predecessor node into a routing table, thepredecessor node preceding a current node relative to the current nodein a first direction of a sorted linked list (act 901). For example, thenode having ID 50 can be inserted into the routing table as apredecessor for the node having ID 64 (the current node). Moving in aclockwise direction 321 (from end A of sorted linked list 304 towardsend B of sorted linked list 304), the node having ID 50 precedes thenode having ID 64. Inserting a predecessor node can establish asymmetric partnership between the current node and the predecessor nodesuch that current node is a partner of predecessor node and thepredecessor node is a partner of the current node

Method 900 includes an act of inserting a successor node into therouting table, the successor node succeeding the current node relativeto the current node in the first direction in the sorted linked list(act 902). For example, the node having ID 76 can be inserted into therouting table as a successor for the node having ID 64 (the currentnode). Moving in a counter-clockwise direction 322, the node having ID76 succeeds the node having ID 64. Inserting a successor node canestablish a symmetric partnership between the current node and thesuccessor node such that current node is a partner of the successor nodeand the successor node is a partner of the current node.

Method 900 includes an act of inserting appropriate neighborhood nodesinto the routing table, the neighborhood nodes identified from thesorted linked list in both the first direction and in a second oppositedirection based on a neighborhood range and neighborhood size (act 903).For example, the nodes having IDs 83, 76, 50, and 46 can be insertedinto the routing table as neighborhood nodes for the node having ID 64(the current node). Based on a neighborhood range of 20 and aneighborhood size 4, the nodes having IDs 83 and 76 can be identified inclockwise direction 321 and the nodes having IDs 50 and 46 can beidentified in counter-clockwise direction 322 (moving from end B ofsorted linked list 304 towards end A of sorted linked list 304). It maybe that in some environments no appropriate neighborhood nodes areidentified. Inserting a neighborhood node can establish a symmetricpartnership between the current node and the neighborhood node such thatcurrent node is a partner of the neighborhood node and the neighborhoodnode is a partner of the current node.

Method 900 includes an act of inserting appropriate routing nodes intothe routing table, the routing nodes identified from the sorted linkedlist in both the first and second directions based on the a number baseand field size of the ID space for the federation infrastructure, therouting nodes representing a logarithmic index of the sorted link listin both the first and second directions (act 904). For example, thenodes having IDs 200, 2, 30, 46, 50, 64, 64, 64, 64, 64, 76, 83, 98, 135and 200 can be inserted into the routing table as routing nodes for thenode having ID 64. Based on the number base 2 and field size of 8 thenodes having IDs 64, 64, 76, 83, 98, 135 and 200 can be identified indirection 321 and the nodes having IDs 64, 64, 50, 46, 30, 2, and 200can be identified in direction 322. As depicted inside ring 306, therouting nodes represent a logarithmic index of the sorted link list 304in both clockwise direction 321 and counter-clockwise direction 322.Inserting a routing node can establish a symmetric partnership betweenthe current node and the routing node such that current node is apartner of the routing node and the routing node is a partner of thecurrent node.

FIG. 7 illustrates an example flow chart of a method 700 for populatinga node routing table that takes proximity criteria into account. Themethod 700 will be described with respect to the rings in FIG. 5. Method700 includes an act of inserting a predecessor node for eachhierarchically partitioned routing ring the current node participates ininto a routing table (act 701). Each predecessor node precedes thecurrent node in a first direction (e.g., clockwise) within eachhierarchically partitioned routing ring the current node participatesin. The hierarchically partitioned routing rings are partitioned inaccordance with corresponding proximity criteria and contain at leastsubsets of a bi-directionally linked list (and possibly the wholebi-directionally linked list). For example, it may be that a specifiednode participates in root ring 501 and sub-rings 511, 522, 523, 531, and542. Thus, a predecessor node is selected for the specified node fromwithin each of the rings 501 and sub-rings 511, 522, 523, 531, and 542.

Method 700 includes an act of inserting a successor node for eachhierarchically partitioned routing ring the current node participates ininto the routing table (act 702). Each successor node succeeding thecurrent node in the first direction within each hierarchicallypartitioned routing ring the current node participates in. For example,a successor node is selected for the specified node from within each ofthe rings 501 and sub-rings 511, 522, 523, 531, and 542.

Method 700 includes an act of inserting appropriate neighborhood nodesfor each hierarchically partitioned routing ring the current nodeparticipates in into the routing table (act 703). The neighborhood nodescan be identified in both the first direction (e.g., clockwise) and in asecond opposite direction (e.g., counter clockwise) based on aneighborhood range and neighborhood size from the hierarchicallypartitioned routing rings the current node participates in. For example,neighborhood nodes can be identified for the specified node from withineach of the rings 501 and sub-rings 511, 522, 523, 531, and 542.

Method 700 includes an act of inserting appropriate routing nodes foreach hierarchically partitioned routing ring the current nodeparticipates in into the routing table (act 704). For example, routingnodes can be identified for the specified node from within each of therings 501 and sub-rings 511, 522, 523, 531, and 542.

In some embodiments, appropriate routing nodes are inserted for eachproximity ring d except the leaf ring (or leaf rings in embodiments thatutilize aliasing), in which the node Y participates. Appropriate routingnodes can be inserted based on the following expression(s):

if Y.s _(d) .id<Y.id+b ^(i) <Y.s _(d+1) .id is true, then use ring d; or

if Y.p _(d) .id<Y.id−b ^(i) <Y.p _(d+1) .id is true, then use ring d.

If a ring has not been identified in the previous step, use the lead(e.g., ring 501) ring as ring d. Now, ring d is the proximity ring inwhich node Y should look for the routing partner closest to z.

FIG. 10 illustrates an example flow chart of a 1000 method for routing amessage towards a destination node. The method 1000 will be describedwith respect to the sorted linked list 304 and ring 306 in FIG. 3.Method 1000 includes an act of a receiving node receiving a messagealong with a number indicating a destination (act 1001). For example,the node having ID 64 can receive a message indicating a destination of212.

Method 1000 includes an act of determining that the receiving node is atleast one of numerically further from the destination than acorresponding predecessor node and numerically further from thedestination than a corresponding successor node (act 1002). For example,in direction 322, ID 64 is further from destination 212 than ID 50 and,in direction 321, ID 64 is further from destination 212 than ID 76.Method 1000 includes an act of determining that the destination is notwithin a neighborhood set of nodes corresponding to the receiving node(act 1003). For example, the node with ID 64 can determine thatdestination 212 is not within the neighborhood set of 83, 76, 50, and46.

The method 1000 includes an act of identifying an intermediate node froma routing table corresponding to the receiving node, the intermediatenode being numerically closer to the destination than other routingnodes in the corresponding routing table (act 1004). For example, thenode having ID 64 can identify the routing node having ID 200 as beingnumerically closer to destination 212 that other routing nodes. Themethod 1000 includes an act of sending the message to the intermediatenode (act 1005). For example, the node having ID 64 can send the messageto the node having ID 200.

FIG. 11 illustrates an example flow chart of a method 1100 for routing amessage towards a destination node based on proximity criteria. Themethod 1100 will be described with respect to the rings in FIG. 4 andFIG. 5. Method 1100 includes an act of a receiving node receiving amessage along with a number indicating a destination and a proximitycriterion (act 1101). The proximity criterion defines one or moreclasses of nodes. The receiving node receives the message as part of acurrent class of nodes selected from among the one or more classes ofnodes based on the proximity criterion. For example, the node having ID172 can receive a message indicating a destination of 201 and proximitycriterion indicating that the destination node be part of classesrepresented by ring 401. The node having ID 172 can receive the messageas part of ring 404.

Method 1100 includes an act of determining that the receiving node is atleast one of, numerically further from the destination than acorresponding predecessor node and numerically further from thedestination than a corresponding successor node, among nodes in aselected class of nodes (act 1102). For example, within ring 404, thenode with ID 172 is further from destination 201 than the node having ID174 in the clockwise direction and is further from destination 201 thanthe node having ID 153 in the counterclockwise direction.

Method 1100 includes an act of determining that the destination is notwithin the receiving node's neighborhood set of nodes for any of the oneor more classes of nodes defined by the proximity criterion (act 1103).For example, the node having ID 172 can determine that destination 201is not in a corresponding neighborhood set in ring 404 or in ring 401.

Method 1100 includes an act of identifying an intermediate node from thereceiving node's routing table, the intermediate node being numericallycloser to the destination than other routing nodes in the routing table(act 1104). For example, the node having ID 172 can identify the nodehaving ID 194 as being numerically closer to destination 201 than otherrouting nodes in ring 404. The method 1100 includes an act of sendingthe message to the intermediate node (act 1105). For example, the nodehaving ID 172 can send the received message to the node having ID 194.The node having ID 172 can send the received message to the node havingID 194 to honor a previously defined partially ordered list of proximitycriterion

Node 194 may be as close to destination 201 as is possible within ring404. Thus, proximity can be relaxed just enough to enable furtherrouting towards the destination to be made in ring 401 in the next leg.That is, routing is transitioned from ring 404 to ring 401 since nofurther progress towards the destination can be made on ring 404.Alternately, it may be that the node having ID 201 is within theneighborhood of the node having ID 194 in ring 401 resulting in nofurther routing. Thus, in some embodiments, relaxing proximity criteriato get to the next higher ring is enough to cause further routing.

However, in other embodiments, incremental relaxation of proximitycriteria causing transition to the next higher ring continues untilfurther routing can occur (or until the root ring is encountered). Thatis, a plurality of transitions to higher rings occurs before furtherrouting progress can be made. For example, referring now to FIG. 5, whenno further routing progress can be made on ring 531, proximity criteriamay be relaxed enough to transition to ring 511 or even to root ring501.

FIG. 6 and the following discussion are intended to provide a brief,general description of a suitable computing environment in which theinvention may be implemented. Although not required, the invention willbe described in the general context of computer-executable instructions,such as program modules, being executed by computer systems. Generally,program modules include routines, programs, objects, components, datastructures, and the like, which perform particular tasks or implementparticular abstract data types. Computer-executable instructions,associated data structures, and program modules represent examples ofthe program code means for executing acts of the methods disclosedherein.

With reference to FIG. 6, an example system for implementing theinvention includes a general-purpose computing device in the form ofcomputer system 620, including a processing unit 621, a system memory622, and a system bus 623 that couples various system componentsincluding the system memory 622 to the processing unit 621. Processingunit 621 can execute computer-executable instructions designed toimplement features of computer system 620, including features of thepresent invention. The system bus 623 may be any of several types of busstructures including a memory bus or memory controller, a peripheralbus, and a local bus using any of a variety of bus architectures. Thesystem memory includes read only memory (“ROM”) 624 and random accessmemory (“RAM”) 625. A basic input/output system (“BIOS”) 626, containingthe basic routines that help transfer information between elementswithin computer system 620, such as during start-up, may be stored inROM 624.

The computer system 620 may also include magnetic hard disk drive 627for reading from and writing to magnetic hard disk 639, magnetic diskdrive 628 for reading from or writing to removable magnetic disk 629,and optical disk drive 630 for reading from or writing to removableoptical disk 631, such as, or example, a CD-ROM or other optical media.The magnetic hard disk drive 627, magnetic disk drive 628, and opticaldisk drive 630 are connected to the system bus 623 by hard disk driveinterface 632, magnetic disk drive-interface 633, and optical driveinterface 634, respectively. The drives and their associatedcomputer-readable media provide nonvolatile storage ofcomputer-executable instructions, data structures, program modules, andother data for the computer system 620. Although the example environmentdescribed herein employs magnetic hard disk 639, removable magnetic disk629 and removable optical disk 631, other types of computer readablemedia for storing data can be used, including magnetic cassettes, flashmemory cards, digital versatile disks, Bernoulli cartridges, RAMs, ROMs,and the like.

Program code means comprising one or more program modules may be storedon hard disk 639, magnetic disk 629, optical disk 631, ROM 624 or RAM625, including an operating system 635, one or more application programs636, other program modules 637, and program data 638. A user may entercommands and information into computer system 620 through keyboard 640,pointing device 642, or other input devices (not shown), such as, forexample, a microphone, joy stick, game pad, scanner, or the like. Theseand other input devices can be connected to the processing unit 621through input/output interface 646 coupled to system bus 623.Input/output interface 646 logically represents any of a wide variety ofdifferent interfaces, such as, for example, a serial port interface, aPS/2 interface, a parallel port interface, a Universal Serial Bus(“USB”) interface, or an Institute of Electrical and ElectronicsEngineers (“IEEE”) 1394 interface (i.e., a FireWire interface), or mayeven logically represent a combination of different interfaces.

A monitor 647 or other display device is also connected to system bus623 via video interface 648. Speakers 669 or other audio output deviceis also connected to system bus 623 via audio interface 649. Otherperipheral output devices (not shown), such as, for example, printers,can also be connected to computer system 620.

Computer system 620 is connectable to networks, such as, for example, anoffice-wide or enterprise-wide computer network, a home network, anintranet, and/or the Internet. Computer system 620 can exchange datawith external sources, such as, for example, remote computer systems,remote applications, and/or remote databases over such networks.

Computer system 620 includes network interface 653, through whichcomputer system 620 receives data from external sources and/or transmitsdata to external sources. As depicted in FIG. 6, network interface 653facilitates the exchange of data with remote computer system 683 vialink 651. Network interface 653 can logically represent one or moresoftware and/or hardware modules, such as, for example, a networkinterface card and corresponding Network Driver Interface Specification(“NDIS”) stack. Link 651 represents a portion of a network (e.g., anEthernet segment), and remote computer system 683 represents a node ofthe network.

Likewise, computer system 620 includes input/output interface 646,through which computer system 620 receives data from external sourcesand/or transmits data to external sources. Input/output interface 646 iscoupled to modem 654 (e.g., a standard modem, a cable modem, or digitalsubscriber line (“DSL”) modem) via link 659, through which computersystem 620 receives data from and/or transmits data to external sources.As depicted in FIG. 6, input/output interface 646 and modem 654facilitate the exchange of data with remote computer system 693 via link652. Link 652 represents a portion of a network and remote computersystem 693 represents a node of the network.

While FIG. 6 represents a suitable operating environment for the presentinvention, the principles of the present invention may be employed inany system that is capable of, with suitable modification if necessary,implementing the principles of the present invention. The environmentillustrated in FIG. 6 is illustrative only and by no means representseven a small portion of the wide variety of environments in which theprinciples of the present invention may be implemented.

In accordance with the present invention, nodes, application layers, andother lower layers, as well as associated data, including routing tablesand node IDs may be stored and accessed from any of thecomputer-readable media associated with computer system 620. Forexample, portions of such modules and portions of associated programdata may be included in operating system 635, application programs 636,program modules 637 and/or program data 638, for storage in systemmemory 622.

When a mass storage device, such as, for example, magnetic hard disk639, is coupled to computer system 620, such modules and associatedprogram data may also be stored in the mass storage device. In anetworked environment, program modules depicted relative to computersystem 620, or portions thereof, can be stored in remote memory storagedevices, such as, system memory and/or mass storage devices associatedwith remote computer system 683 and/or remote computer system 693.Execution of such modules may be performed in a distributed environmentas previously described.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Allchanges, which come within the meaning and range of equivalency of theclaims, are to be embraced within their scope.

1. In a federation infrastructure, a method for creating and maintaininga node distribution data structure, the method comprising: an act ofaccessing from a current node at least one routing table in thefederation infrastructure to determine identification information foreach node in the federation infrastructure of which the routing tablehas knowledge, wherein one or more of the nodes in the routing tablereferences at least one other node in the federation infrastructure; anact of subsequently accessing the other nodes in the federationinfrastructure by following the nodes pointed to by each node; and basedon the accessed nodes, an act of forming a node distribution datastructure that points to every node in the federation infrastructure. 2.The method as recited in claim 1, wherein the nodes and routing tablesare members of the same proximal ring.
 3. The method as recited in claim2, wherein at least the node identification information and any nodereferences maintained in the routing table are used to address a messageto at least one other node in the federation infrastructure.
 4. Themethod as recited in claim 2, wherein at least the node identificationinformation and any node references maintained in the routing table areused to determine at least one node from a set of nodes to address amessage to.
 5. The method as recited in claim 3, further comprisingsending a plurality of messages instead of a single message.
 6. Themethod as recited in claim 1, wherein the nodes and routing tables aremembers of different proximal rings.
 7. The method as recited in claim6, wherein at least the node identification information and any nodereferences maintained in the routing table are used to address a messageto at least one other node in the federation infrastructure.
 8. Themethod as recited in claim 6, wherein at least the node identificationinformation and any node references maintained in the routing table areused to determine at least one node from a set of nodes to address amessage to.
 9. The method as recited in claim 1, wherein the nodedistribution data structure comprises a connectivity graph.
 10. In afederation infrastructure, a system configured to create and maintain anode distribution data structure, the system comprising: an accessingmodule configured to access from a current node at least one routingtable in the federation infrastructure to determine identificationinformation for each node in the federation infrastructure of which therouting table has knowledge, wherein one or more of the nodes in therouting table points to at least one other node in the federationinfrastructure; a second accessing module configured to subsequentlyaccess the other nodes in the federation infrastructure by following thenodes pointed to by each node; and a distributed forming moduleconfigured to form, based on the accessed nodes, a node distributiondata structure that points to every node in the federationinfrastructure.
 11. The system as recited in claim 10, wherein the nodesand routing tables are members of the same proximal ring.
 12. The systemas recited in claim 11, wherein at least the node identificationinformation and any node references maintained in the routing table areused to address a message to at least one other node in the federationinfrastructure.
 13. The system as recited in claim 11, wherein at leastthe node identification information and any node references maintainedin the routing table are used to determine at least one node from a setof nodes to address a message to.
 14. The system as recited in claim 12,further comprising sending a plurality of messages instead of a singlemessage.
 15. The system as recited in claim 10, wherein the nodes androuting tables are members of different proximal rings.
 16. The systemas recited in claim 15, wherein at least the node identificationinformation and any node references maintained in the routing table areused to address a message to at least one other node in the federationinfrastructure.
 17. The system as recited in claim 15, wherein at leastthe node identification information and any node references maintainedin the routing table are used to determine at least one node from a setof nodes to address a message to.
 18. The system as recited in claim 10,wherein the node distribution data structure comprises a connectivitygraph.
 19. In a federation infrastructure, a system including a fullyconnected node distribution data structure, the system comprising: areceiving module for receiving at a current node an indication that amessage is to be sent to from the current node to each of the nodes ofthe federation infrastructure; and a sending module for sending themessage to at least one other node in the federation infrastructure,whereupon each receiving node accesses stored state information to sendthe message to at least one other node until all nodes have received themessage at least once.
 20. The system as recited in claim 19, whereinthe message is addressed to a particular identified node among theplurality of nodes in the federation infrastructure.